A Framework for Exploring and Evaluating Mechanics in Human Computation Games

Human computation games (HCGs) are a crowdsourcing approach to solving computationally-intractable tasks using games. In this paper, we describe the need for generalizable HCG design knowledge that accommodates the needs of both players and tasks. We propose a formal representation of the mechanics in HCGs, providing a structural breakdown to visualize, compare, and explore the space of HCG mechanics. We present a methodology based on small-scale design experiments using fixed tasks while varying game elements to observe effects on both the player experience and the human computation task completion. Finally we discuss applications of our framework using comparisons of prior HCGs and recent design experiments. Ultimately, we wish to enable easier exploration and development of HCGs, helping these games provide meaningful player experiences while solving difficult problems.Comment: 11 pages, 5 figure

Evaluating Singleplayer and Multiplayer in Human Computation Games

Human computation games (HCGs) can provide novel solutions to intractable computational problems, help enable scientific breakthroughs, and provide datasets for artificial intelligence. However, our knowledge about how to design and deploy HCGs that appeal to players and solve problems effectively is incomplete. We present an investigatory HCG based on Super Mario Bros. We used this game in a human subjects study to investigate how different social conditions---singleplayer and multiplayer---and scoring mechanics---collaborative and competitive---affect players' subjective experiences, accuracy at the task, and the completion rate. In doing so, we demonstrate a novel design approach for HCGs, and discuss the benefits and tradeoffs of these mechanics in HCG design.Comment: 10 pages, 4 figures, 2 table

Design and Evaluation of Intelligent Reward Structures in Human Computation Games

Despite the ubiquity of artificial intelligence, some problems and procedures— such as building commonsense knowledge understanding or generating creative works— have no or few effective algorithmic solutions, yet are considered straightforward for humans to solve. Human computation games (HCGs) are playful, game-based interfaces for tackling these problems through crowdsourcing. HCGs have been used to solve tasks that were and still are considered complex for computational algorithms such as image tagging, protein synthesis, 3D structure reconstruction, and creative artifact generation. However, despite these successes, HCGs have not seen broad adoption compared to other types of serious digital games. Among the many reasons for this lack of adoption is the reality that these games are typically not seen as engaging or compelling to play, as well as the fact that creating HCGs comes at a high development cost to task providers who are typically not game development experts. This thesis is a step towards building and establishing a more formalized design understanding of how to create HCGs that both provide a compelling player experience and complete the underlying task effectively. In this thesis, I explore reward mechanics in HCGs. Reward mechanics are integral to HCGs due their associations with player motivation, compensation, and task validation. I first propose a framework for understanding HCG mechanics and advocate for an experimental methodology evaluating both player experience and task completion metrics to understand variations in HCG mechanics. I then use these tools to frame and design three experiments that explore small-scale variations of reward systems in HCGs: reward functions, reward distribution, and reward personalization. These studies demonstrate that even small variations in rewards (i.e., offering players the ability to choose the type of reward) may have significant positive effects on both player experience and task completion metrics. I also show that some variations (i.e., co-located, competitive reward scoring) may have both positive and negative tradeoffs across these metrics. Moreover, this work observes that existing, anecdotal design wisdom for HCGs may not always hold (i.e., allowing players to verbally collude actually predicts higher task solution accuracy). Altogether, this thesis demonstrates that certain aspects of reward systems in HCGs can be varied to improve the player experience without compromising task completion metrics, and builds more empirically-tested design knowledge for creating more engaging, effective HCGs.Ph.D

Social gamification in enterprise crowdsourcing

Enterprise crowdsourcing capitalises on the availability of employ-ees for in-house data processing. Gamification techniques can help aligning employees’ motivation to the crowdsourcing endeavour. Although hitherto, research efforts were able to unravel the wide arsenal of gamification techniques to construct engagement loops, little research has shed light into the social game dynamics that those foster and how those impact crowdsourcing activities. This work reports on a study that involved 101 employees from two multinational enterprises. We adopt a user-centric approach to ap-ply and experiment with gamification for enterprise crowdsourcing purposes. Through a qualitative study, we highlight the importance of the competitive and collaborative social dynamics within the enterprise. By engaging the employees with a mobile crowdsourc-ing application, we showcase the effectiveness of competitiveness towards higher levels of engagement and quality of contributions. Moreover, we underline the contradictory nature of those dynam-ics, which combined might lead to detrimental effects towards the engagement to crowdsourcing activities

Potential Pitfalls in Pre-implantation Genetic Diagnosis in a Patient with Tuberous Sclerosis and Isolated Mosaicism for a TSC2 Variant in Renal Tissue

Tuberous sclerosis complex (TSC) is an autosomal dominant disorder that displays a wide spectrum of clinical manifestations, often affecting multiple organs including the kidneys, brain, lungs, and skin. A pathogenic mutation in either the TSC1 or TSC2 gene can be detected in almost 85% of the cases, with mosaicism accounting for about half of the remaining cases. We report a case of TSC diagnosed clinically, requesting genetic counselling regarding reproductive risks. No mutation was identified on initial testing of peripheral blood; however, mosaicism for a likely pathogenic frameshift variant in TSC2 was detected at a level of 15% in renal angiomyolipoma tissue. Despite widespread clinical manifestations of TCS, this variant was not detected in skin fibroblasts or saliva, raising the possibility this is an isolated somatic mutation in renal tissue with the underlying germline mutation not yet identified. This case highlights the difficulties when counselling patients with mosaicism regarding their reproductive risks and prenatal diagnostic options

Truncating SRCAP variants outside the Floating-Harbor syndrome locus cause a distinct neurodevelopmental disorder with a specific DNA methylation signature

Truncating variants in exons 33 and 34 of the SNF2-related CREBBP activator protein (SRCAP) gene cause the neurodevelopmental disorder (NDD) Floating-Harbor syndrome (FLHS), characterized by short stature, speech delay, and facial dysmorphism. Here, we present a cohort of 33 individuals with clinical features distinct from FLHS and truncating (mostly de novo) SRCAP variants either proximal (n = 28) or distal (n = 5) to the FLHS locus. Detailed clinical characterization of the proximal SRCAP individuals identified shared characteristics: developmental delay with or without intellectual disability, behavioral and psychiatric problems, non-specific facial features, musculoskeletal issues, and hypotonia. Because FLHS is known to be associated with a unique set of DNA methylation (DNAm) changes in blood, a DNAm signature, we investigated whether there was a distinct signature associated with our affected individuals. A machine-learning model, based on the FLHS DNAm signature, negatively classified all our tested subjects. Comparing proximal variants with typically developing controls, we identified a DNAm signature distinct from the FLHS signature. Based on the DNAm and clinical data, we refer to the condition as "non-FLHS SRCAP-related NDD.'' All five distal variants classified negatively using the FLHS DNAm model while two classified positively using the proximal model. This suggests divergent pathogenicity of these variants, though clinically the distal group presented with NDD, similar to the proximal SRCAP group. In summary, for SRCAP, there is a clear relationship between variant location, DNAm profile, and clinical phenotype. These results highlight the power of combined epigenetic, molecular, and clinical studies to identify and characterize genotype-epigenotype-phenotype correlations

Defining the genotypic and phenotypic spectrum of X-linked MSL3-related disorder

Purpose We sought to delineate the genotypic and phenotypic spectrum of female and male individuals with X-linked, MSL3-related disorder (Basilicata-Akhtar syndrome). Methods Twenty-five individuals (15 males, 10 females) with causative variants in MSL3 were ascertained through exome or genome sequencing at ten different sequencing centers. Results We identified multiple variant types in MSL3 (ten nonsense, six frameshift, four splice site, three missense, one in-frame-deletion, one multi-exon deletion), most proven to be de novo, and clustering in the terminal eight exons suggesting that truncating variants in the first five exons might be compensated by an alternative MSL3 transcript. Three-dimensional modeling of missense and splice variants indicated that these have a deleterious effect. The main clinical findings comprised developmental delay and intellectual disability ranging from mild to severe. Autism spectrum disorder, muscle tone abnormalities, and macrocephaly were common as well as hearing impairment and gastrointestinal problems. Hypoplasia of the cerebellar vermis emerged as a consistent magnetic resonance image (MRI) finding. Females and males were equally affected. Using facial analysis technology, a recognizable facial gestalt was determined. Conclusion Our aggregated data illustrate the genotypic and phenotypic spectrum of X-linked, MSL3-related disorder (Basilicata-Akhtar syndrome). Our cohort improves the understanding of disease related morbidity and allows us to propose detailed surveillance guidelines for affected individuals

Defining the genotypic and phenotypic spectrum of X-linked MSL3-related disorder

PURPOSE: We sought to delineate the genotypic and phenotypic spectrum of female and male individuals with X-linked, MSL3-related disorder (Basilicata-Akhtar syndrome). METHODS: Twenty-five individuals (15 males, 10 females) with causative variants in MSL3 were ascertained through exome or genome sequencing at ten different sequencing centers. RESULTS: We identified multiple variant types in MSL3 (ten nonsense, six frameshift, four splice site, three missense, one in-frame-deletion, one multi-exon deletion), most proven to be de novo, and clustering in the terminal eight exons suggesting that truncating variants in the first five exons might be compensated by an alternative MSL3 transcript. Three-dimensional modeling of missense and splice variants indicated that these have a deleterious effect. The main clinical findings comprised developmental delay and intellectual disability ranging from mild to severe. Autism spectrum disorder, muscle tone abnormalities, and macrocephaly were common as well as hearing impairment and gastrointestinal problems. Hypoplasia of the cerebellar vermis emerged as a consistent magnetic resonance image (MRI) finding. Females and males were equally affected. Using facial analysis technology, a recognizable facial gestalt was determined. CONCLUSION: Our aggregated data illustrate the genotypic and phenotypic spectrum of X-linked, MSL3-related disorder (Basilicata-Akhtar syndrome). Our cohort improves the understanding of disease related morbidity and allows us to propose detailed surveillance guidelines for affected individuals

The clinical and genetic spectrum of autosomal-recessive TOR1A-related disorders.

In the field of rare diseases, progress in molecular diagnostics led to the recognition that variants linked to autosomal-dominant neurodegenerative diseases of later onset can, in the context of biallelic inheritance, cause devastating neurodevelopmental disorders and infantile or childhood-onset neurodegeneration. TOR1A-associated arthrogryposis multiplex congenita 5 (AMC5) is a rare neurodevelopmental disorder arising from biallelic variants in TOR1A, a gene that in the heterozygous state is associated to torsion dystonia-1 (DYT1 or DYT-TOR1A), an early-onset dystonia with reduced penetrance. While 15 individuals with TOR1A-AMC5 have been reported (less than 10 in detail), a systematic investigation of the full disease-associated spectrum has not been conducted. Here, we assess the clinical, radiological and molecular characteristics of 57 individuals from 40 families with biallelic variants in TOR1A. Median age at last follow-up was 3 years (0-24 years). Most individuals presented with severe congenital flexion contractures (95%) and variable developmental delay (79%). Motor symptoms were reported in 79% and included lower limb spasticity and pyramidal signs, as well as gait disturbances. Facial dysmorphism was an integral part of the phenotype, with key features being a broad/full nasal tip, narrowing of the forehead and full cheeks. Analysis of disease-associated manifestations delineated a phenotypic spectrum ranging from normal cognition and mild gait disturbance to congenital arthrogryposis, global developmental delay, intellectual disability, absent speech and inability to walk. In a subset, the presentation was consistent with fetal akinesia deformation sequence with severe intrauterine abnormalities. Survival was 71% with higher mortality in males. Death occurred at a median age of 1.2 months (1 week - 9 years) due to respiratory failure, cardiac arrest, or sepsis. Analysis of brain MRI studies identified non-specific neuroimaging features, including a hypoplastic corpus callosum (72%), foci of signal abnormality in the subcortical and periventricular white matter (55%), diffuse white matter volume loss (45%), mega cisterna magna (36%) and arachnoid cysts (27%). The molecular spectrum included 22 distinct variants, defining a mutational hotspot in the C-terminal domain of the Torsin-1A protein. Genotype-phenotype analysis revealed an association of missense variants in the 3-helix bundle domain to an attenuated phenotype, while missense variants near the Walker A/B motif as well as biallelic truncating variants were linked to early death. In summary, this systematic cross-sectional analysis of a large cohort of individuals with biallelic TOR1A variants across a wide age-range delineates the clinical and genetic spectrum of TOR1A-related autosomal-recessive disease and highlights potential predictors for disease severity and survival

Creative destruction in science

Drawing on the concept of a gale of creative destruction in a capitalistic economy, we argue that initiatives to assess the robustness of findings in the organizational literature should aim to simultaneously test competing ideas operating in the same theoretical space. In other words, replication efforts should seek not just to support or question the original findings, but also to replace them with revised, stronger theories with greater explanatory power. Achieving this will typically require adding new measures, conditions, and subject populations to research designs, in order to carry out conceptual tests of multiple theories in addition to directly replicating the original findings. To illustrate the value of the creative destruction approach for theory pruning in organizational scholarship, we describe recent replication initiatives re-examining culture and work morality, working parents\u2019 reasoning about day care options, and gender discrimination in hiring decisions. Significance statement It is becoming increasingly clear that many, if not most, published research findings across scientific fields are not readily replicable when the same method is repeated. Although extremely valuable, failed replications risk leaving a theoretical void\u2014 reducing confidence the original theoretical prediction is true, but not replacing it with positive evidence in favor of an alternative theory. We introduce the creative destruction approach to replication, which combines theory pruning methods from the field of management with emerging best practices from the open science movement, with the aim of making replications as generative as possible. In effect, we advocate for a Replication 2.0 movement in which the goal shifts from checking on the reliability of past findings to actively engaging in competitive theory testing and theory building. Scientific transparency statement The materials, code, and data for this article are posted publicly on the Open Science Framework, with links provided in the article